library(readr)
library(tidyverse)
library(data.table)
library(kableExtra)
library(readxl)
library(plotly)
library(here)

Taking Images

The images are taken with the 360-degree camera RICOH THETA S (see here). The camera has two inbuilt lenses, producing a 360-degree image.

  • At each plot 4 images are made.
  • One image directly at the centre of the plot.
  • Three images around the centre at a distance of 15m distributed evenly in all directlions (0, 120, 240 degrees).

To capture the sorrounding as good as possible and to cover the hemisphere in all directions around the plot centre, the camera is held vertically above the head.

Naming of the images should be consistent as CityClusterPlotnumber_Imagenumber (e.g. MDLDT01_01).

knitr::include_graphics(here("raw_data/img/plotdesign.png"))
Plotdesign

Plotdesign

Image Analysis

The images are converted into binary images and gap fraction is calculated automatically using ImageJ and the hemispherical_2.0 macro.

Software

ImageJ is a Java-based image processing program developed at the National Institutes of Health and the Laboratory for Optical and Computational Instrumentation (LOCI, University of Wisconsin). It is open-source and can be downloaded for free here.
Download ImageJ

Hemispherical_2.0 is a macro for ImageJ that batch processes large quantities of both digital hemispherical and non-hemispherical canopy photographs at comparatively faster computational speeds. It was developed at the Institute of Forest Inventory and Remote Sensing in Goettigen and can be downloaded here.
Download Hemispherical_2.0

The manual is included in the download above and can also be seen seperately here.

Processing

For processing the images with ImageJ and the hemispherical_2.0 macro the images have to be masked in a first step. Then they are converted to binary (sky / no sky) images and gap statistics are derived.

Workflow in ImageJ:

  1. Clicking on Plugins -> Hemispherical_2.0
  2. select the input folder containing the hemispherical photos
  3. select the output folder where processed images should be stored
  4. click rectangular selection tool and select image part to be processed.
    (In our case we choose the upper half of the first image since we are interested in the sky/canopy and not the ground)
  5. click Ok to start batch processing
    (this might not work with several hundred images, in this case process the images in chunks)
  6. save results file
  7. finished

Results

Visual Comparison

before

knitr::include_graphics(here("raw_data/img/MDLDT04_1_xmp_e.jpg"))
RGB-Image in flat projection as taken in the field

RGB-Image in flat projection as taken in the field

after

knitr::include_graphics(here("output_data/img/MDLDT04_1_xmp_e_T.jpg"))
Binary Black and White Image after processing

Binary Black and White Image after processing

Numerical Results

results_MDL <- read_csv(here("output_data/doc/hemiphotos_results_MDL.csv"))

results_MAW <- read_csv(here("output_data/doc/hemiphotos_results_MAW.csv"))

results_MGZ <- read_csv(here("output_data/doc/hemiphotos_results_MGZ.csv"))

# merge data
hemi <- bind_rows(results_MDL, results_MAW, results_MGZ)

# get plot data from image name
hemi <- hemi %>%
  mutate(city = as.factor(substr(photo, start = 1, stop = 3)),
         cluster = as.factor(substr(photo, start = 4, stop = 5)),
         plot = as.factor(substr(photo, start = 6, stop = 7))) %>%
  # change cluster levels to be more meaningful and to be compatible with bird data
  mutate(cluster = recode(cluster, RP = "Paddy field", MH = "Hillside", DT = "Downtown", UC= "University campus"))
# create color scale to map levels consistent in plots
myColors <- scales::viridis_pal(option= "D")(nlevels(hemi$cluster))
names(myColors) <- levels(hemi$cluster)
colscale <- scale_colour_manual(name = "grp",values = myColors)

Output table from Hemispherical_2.0

head(hemi) %>% 
  select(-city, -cluster, -plot) %>%
  kable(escape = F) %>%
  kable_styling(bootstrap_options = c("striped", "hover", "condensed", "responsive"), font_size = 12, fixed_thead = T)
Index photo no_of_gaps gap_area gap_fraction
0 MDLDT01_1_xmp_e.jpg 974 3350562 43.036
1 MDLDT01_2_xmp_e.jpg 2817 1063230 13.657
2 MDLDT01_3_xmp_e.jpg 1621 4741994 60.908
3 MDLDT01_4_xmp_e.jpg 612 4207448 54.042
4 MDLDT02_1_xmp_e.jpg 1219 4710816 60.508
5 MDLDT02_2_xmp_e.jpg 1193 4571665 58.720

Simple Analysis Example

Output data can be analyzed and plotted by city and cluster for example.

Average gap fraction per city and cluster

The gap fraction is a metric for the openness of the plots, hence plots in the city or in forested sites should have lower gap fractions than agricultural dominated sites

hemi %>%
  plot_ly(y = ~gap_fraction, 
          color = ~cluster,
          colors = myColors, 
          type = "box")
Average number of gaps per city and cluster

The number of gaps might be an indicator for structural diversity. As this is dependend on a lot of factors this metric should be interpreted carefully.

hemi %>%
  plot_ly(y = ~no_of_gaps, 
          color = ~cluster,
          colors = myColors, 
          type = "box")

Combination with bird data

Do relations between bird count data and hemispherical photos exist?

# read bird data
birddata <- read_excel(path= here("raw_data/doc/birds-all.xlsx"), sheet = "ALL")

# group hemispherical data by plot
hemi.grouped <- hemi %>%
  group_by(city, cluster, plot) %>%
  summarise(no_of_images= n(),
            mean_gap_fraction = mean(gap_fraction, na.rm = T),
         mean_no_of_gaps = mean(no_of_gaps, na.rm = T),
         mean_gap_area = mean(gap_area, na.rm = T))

# get plot data from image name
birddata <- birddata %>%
  select(Country, City, Habitat, Point, English_Name, Count) %>%
  mutate(city = as.factor(substr(Point, start = 1, stop = 3)),
         cluster = as.factor(substr(Point, start = 4, stop = 5)),
         plot = as.factor(substr(Point, start = 6, stop = 7))) %>%
  # change cluster levels to be more meaningful and to be compatible with hemispherical data
  mutate(cluster = recode(cluster, PF = "Paddy field", MH = "Hillside", DT = "Downtown", UC= "University campus"))

# merge bird data and hemispherical data
hemibird <- merge(hemi.grouped, birddata, by=c("city","cluster", "plot"))
Number of bird counts by gap fraction
hemibird %>%
  group_by(city, cluster, plot) %>%
  summarise(no_of_species = n_distinct(English_Name),
            bird_count = sum(Count),
            mean_no_of_gaps = mean(mean_no_of_gaps, na.rm = T),
            mean_gap_area = mean(mean_gap_area, na.rm = T),
            mean_gap_fraction = mean(mean_gap_fraction, na.rm = T)) %>%
  
  plot_ly(x= ~mean_gap_fraction, 
          y= ~bird_count, 
          color= ~cluster,
          colors = myColors,
           # Hover text:
          text = ~paste("Number of birds: ", bird_count, 
                        "<br>Number of species:", no_of_species, 
                        "<br>Gap fraction:", mean_gap_fraction, 
                        "<br>Number of gaps:", mean_no_of_gaps)) %>%
  add_markers()
Number of bird species by number of gaps
hemibird %>%
  group_by(city, cluster, plot) %>%
  summarise(no_of_species = n_distinct(English_Name),
            bird_count = sum(Count),
            mean_no_of_gaps = mean(mean_no_of_gaps, na.rm = T),
            mean_gap_area = mean(mean_gap_area, na.rm = T),
            mean_gap_fraction = mean(mean_gap_fraction, na.rm = T)) %>%
  
  plot_ly(x= ~mean_no_of_gaps, 
          y= ~no_of_species,
          color= ~cluster,
          colors = myColors,
           # Hover text:
          text = ~paste("Number of birds: ", bird_count, 
                        "<br>Number of species:", no_of_species, 
                        "<br>Gap fraction:", mean_gap_fraction, 
                        "<br>Number of gaps:", mean_no_of_gaps)) %>%
  add_markers()

Combination with other data

In a further step this data can be merged with observations from the field or remote sensing data to enable a deeper anaylsis.

# read remote sensing variables
rs <- read_csv(file = here("raw_data/doc/remote-sensing-variables.csv"))

# get plot data from image name
rs <- rs %>%
  filter(buffer_radius == 125) %>%
  mutate(city = as.factor(substr(plot_id, start = 1, stop = 3)),
         cluster = as.factor(substr(plot_id, start = 4, stop = 5)),
         plot = as.factor(substr(plot_id, start = 6, stop = 7))) %>%
  # change cluster levels to be more meaningful and to be compatible with hemispherical data
  mutate(cluster = recode(cluster, RP = "Paddy field", MH = "Hillside", DT = "Downtown", UC= "University campus"))

birddata_grouped <- birddata %>%
  group_by(city, cluster, plot) %>%
  summarise(no_of_species = n_distinct(English_Name),
            bird_count = sum(Count))

# merge bird data and hemispherical data
rsbird <- merge(rs, birddata_grouped, by=c("city","cluster", "plot"))
Mean NDVI per cluster

(125m radius around plot center)

rsbird %>%
  plot_ly(y = ~NDVI_mean, 
          color = ~cluster,
          colors = myColors, 
          type = "box")
Number of bird species by mean NDVI
rsbird %>%
  plot_ly(x= ~NDVI_mean, 
          y= ~no_of_species,
          color= ~cluster,
          colors = myColors,
           # Hover text:
          text = ~paste("Number of birds: ", bird_count, 
                        "<br>Number of species:", no_of_species)) %>%
  add_markers()
 

A work by Jens Wiesehahn